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Abstract 

The  increasing  cost  of  Department  of  Defense  acquisitions  combined  with 
the  shrinking  Defense  budget  puts  a  premium  on  continuing  the  funding  of  only 
those  programs  that  are  practical  and  affordable.  This  emphasizes  a  need  for 
good  methods  of  monitoring  the  progress  of  programs.  Current  methods  of 
controlling  and  evaluating  cost  performance  are  limited. 

It  is  our  belief  that  the  generation  of  S-curves  using  current  Cost  and 
Schedule  Control  Systems  Criteria  (C/SCSC)  data  will  provide  a  mathematical 
and/or  graphical  approach  to  assist  in  the  control  of  program  costs.  This  thesis 
attempted  to  plot  the  percentage  of  time  for  a  contract's  comploticn  against  the 
percentage  of  actual  cost  spent  during  that  period.  Each  contract  was  separated 
into  a  sub^tegory,  averaged  ^d  then  plotted.  This  developed  a  signature 
curve  for  categories  of  contracts. 

Using  the  signature  S-ouves.  a  program  manager  can  compare  their 
program's  percentile  complete  to  percentage  of  cost  to  the  standard  for  similar 
programs  and  determine  if  their  program  is  on  track. 

This  thesis  did  not  result  in  the  distinctive  S  cun/e  pattern  anticipated  for 
an  average  of  program  contracts  for  various  categories.  The  averaging  of  the 
contracts  resulted  in  a  smoothing  of  the  S-curve  into  a  nearly  linear  pattern.  The 
resulting  standard  average  oirve  for  a  category  of  programs  would  provide  little 
irtsight  into  the  ^tus  of  a  single  program  when  used  in  comparison  to  that 
program. 
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EVALUATION  OF  MAJOR  DEPARTMENT  OF  DEFENSE  PURCHASES 


L  Introduction 


Overview 

One  of  the  strongest  and  most  consistent  trends  in  the  Department  of 
Defense  over  the  last  30  years  has  been  the  spiraling  costs  of  weapon 
systems  (1 .218).  These  huge  increases  in  the  cost  of  defense  have  resulted  in 
an  atmosphere  of  skepticism  among  the  general  population  regarding  the 
efficiency  the  d^ense  department,  particularly  in  their  procurement  policies. 
Headlines  highlighting  stories  of  $437  hammers  and  $3,046  crash-proof  coffee 
pots  are  rampant  (2.31 ).  In  order  to  curtail  the  spiraling  a>st  of  weapon 
systems  and  eliminate  the  tales  of  overpriced  merchandise  the  Department  of 
Defense  in  the  late  60's  instituted  an  extensive  ov^haul  of  its  procurement 
system  that  was  centered  on  the  Cost/Schedule  (Control  Systems  Criteria. 
Department  of  Defense  Instruction  7000.2,  implemented  December  22, 1967 
defined  35  oiteria  that  all  contractor  management  control  systems  were 
required  to  meet  before  their  systems  could  be  used  on  government  contracts. 
The  idea  was  to  standardize  the  various  contractors'  systems  data  provided  to 
government  program  offices  for  the  monitoring  of  procurement  contracts  and 
controlling  the  cost  of  weapon  systems. 

Today  the  program  offices  are  awash  in  valid  data  provided  by  the 
conirador.  Additionally,  a  variety  d  methodologies  are  currently  used  to 
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analyze  contractor  performance.  Yet,  program  cost  growth  continues  to  remain 
uncontrolled  (1 ;  1 5).  It  is  possible  that  a  contributing  factor  to  the  problem  is 
the  lack  of  standardized  tools  that  structure  the  data  to  Identify  problems  in  the 
programs  early  enough  so  that  they  can  receive  management  attention  and  be 
rectified  (13:18).  The  Department  of  Defense  is  constantly  looking  for  new  and 
more  efficient  ways  of  focusing  on  the  data  provided  to  the  program  offices  so 
that  it  can  be  used  efficiently. 

General  Issue 

A  new  technique  was  recently  developed  and  used  by  the  Office  of  the 
Undersecretary  of  Defense  for  Acquisition  Policy  and  Program  Integration  / 
Performance  Measurement  (OUSO(A)).  This  technique  is  the  calculation  of 
8<curves  for  defense  programs  that  reflect  the  cumulative  financial  status  of  the 
programs  over  time.  The  S-oirve  utilized  by  OUSD{A)  plots  the  cost  of  a  project 
as  measured  by  the  Actual  Cost  of  Work  Performed  (ACWP)  as  a  percentage  of 
the  prelected  total  cost  of  the  program  against  the  percentage  completion  of  the 
program  over  time.  This  technique  was  utilized  by  OUSD(A)  in  evaluating  the 
A-12  aircraft  program  (3:2).  The  A-12  program  was  a  Navy  effort  to  design  and 
build  the  next  generation,  carrier  based  medium  attack  aircraft  to  replace  the 
A-6.  The  A-12  program  was  a  risky  program  because  it  lnt»rporated  new 
composite  material  technology.  Analysis  of  the  program  was  based  on  an  EAC 
methodology  that  cor^stentiy  produced  low  estimates  (3:2).  The  result  was 
prefect  cost  overruns  that  were  nx^t  accurately  estimated,  particularly  early  in 
the  estimate.  Mr.  Christie  and  Mr.  Abba  oi  OUSD(A)  plotted  the  percentage  of 
cost  incurred  against  the  percentage  of  contrad  compietion  for  several  aircraft 
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projects  with  a  resulting  S-curve  profile.  An  S  curve  envelope  was  developed 
which  reflected  the  pattern  for  normal  aircraft  projects.  A  comparison  of  the 
envelope  to  the  A-12  program  revealed  a  significant  divergence  of  this 
program  from  previous  successful  programs  (Figure  1).  This  analytical  tool 
was  instrumental  in  showing  that  the  A-12  program  was  in  trouble  and 
ultimately  contributed  to  the  cancellation  of  the  program. 


Figure  1.  Hand  drawn  S-Curve  developed  by  Mr.  Christie  and  Mr.  Abba. 

This  technique  was  very  persuasive  in  the  A-12  case  and  it  could  be  very  useful 
to  the  project  offices  in  evaluating  the  progress  of  their  current  programs.  Yet  the 
technique  is  currently  not  widely  used  by  the  command  level  project  offices. 
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Specific  Probiem 

Analysis  of  defense  projects  needs  to  be  commonly  applied  and  understood  by 
both  the  project  offices  and  upper  levels  of  command.  If  different  levels  of  the 
defense  organization  are  using  different  measures  of  the  progress  of  a  project,  it  is 
difficult  to  reach  agreement  on  the  overall  status  of  the  project.  Therefore,  it  is 
Important  that  common  analysis  techniques  be  employed  by  both  upper  levels  of  the 
defense  organization  and  the  project  offices.  OUSO(A)  has  developed  and 
successftjlly  utilized  S  curves  in  presenting  analysis  of  airaaft  projects  to  the  Office  of 
the  Secretary  of  Defense  (OSD).  This  technique  needs  to  be  analyzed  and 
understood  by  all  levels  of  the  defense  organisation.  In  order  for  this  tool  to  be 
generally  used  it  needs  to  be  developed  to  the  point  where  it  can  be  easily  applied  by 
the  project  offices.  joi...non  database  and  readily  available  computer  tools  would 
greatly  facilitate  the  use  of  this  technique.  Furthermore,  the  technique  needs  to  be 
extended  to  see  if  it  is  applicable  to  defense  projects  other  than  aircraft  so  that  it  can 
be  broadly  applied  as  a  measurement  tool  across  the  breadth  of  defense  projects. 

Hypothesis 

S  curves  as  an  analytical  tool  have  been  hand-developed  and  applied  to 
specific  projects.  The  object  of  this  thesis  is  to  construct  a  database  that  can 
readily  be  used  to  develop  S-curves  so  that  broad  categories  of  projects  can 
be  compared  and  analyzed  using  S  curves.  The  specific  hypothesis  to  be 
tested  is  that  S-curves  are  applicable  in  the  cost  analysis  of  all  defense 
acquisition  programs.  A  defense  acquisition  program  is  a  directed,  funded 
effort  that  is  designed  to  provide  (the  Department  of  Defense)  a  new  or 
improved  material  capability  in  response  to  a  validated  need  (16:2). 
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investigative  questions  to  be  answered  are; 

1 .  What  research  has  examined  applying  S-curve  theory  to  DOD 

cost  estimating? 

2.  Can  the  Defense  Acquisition  Executive  Summary  (DAES) 

database  be  used  to  produce  S  curves? 

3.  How  is  percentage  of  time  defined  and  derived? 

4.  How  is  percentage  of  cost  defined  and  derived? 

5.  Are  S  curves  different  for  various  categories  of  programs? 

6.  How  can  S-curves  be  used  by  program  managers  and  analysts? 

Scope  of  the  Research  Project 

This  study  was  confined  to  ttie  analysis  of  the  S-curve  technique  in 
measuring  project  status.  The  S-curve  technique  was  applied  to  the  database 
of  over  500  completed  contracts  available  from  Defense  Acquisition  Executive 
Summary  Reports  (20).  This  database  was  reviewed  for  its  applicability  for 
use  in  S  curve  analysis.  Only  sub-categories  of  the  data  that  ointained 
sufficient  data  points  to  develop  an  average  for  the  sub-category  were 
developed.  Programs  osnsidered  hsr  inclusion  in  the  analysis  were  restricted 
to  those  with  enough  history  to  provide  a  reasonable  S-curve  profile.  For  this 
study  this  consisted  of  contracts  that  were  at  least  75%  complete  and  that  had 
data  in  the  database  frcm  the  25%  completion  point.  One  disadvantage  of  this 
selection  criterion  was  that  the  database  only  included  successful  programs. 
Unsuccessful  programs  were  usually  termir^ted  prior  to  the  75%  completion 
point.  This  is  not  perceived  as  s  serious  fiaw,  as  the  intend  of  this  study  was  to 
derive  S  curves  which  model  successful  programs. 


5 


The  following  chapters  describe  the  relevant  literature,  the  methodology 
of  this  study,  the  results  of  the  analysis,  conclusions,  and  recommendations  for 
further  research. 
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II.  Literature  Review 


Cost/Schedule  Control  Systems  Criteria 

The  Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  are  a  set  of 
standards  that  a  contractor's  management  system  must  meet  in  order  for  the 
contractor  to  participate  in  a  major  defense  program.  Basically,  C/SCSC 
ensures  that  the  data  provided  by  contractors  in  their  monthly  cost 
performance  reports  are  accurate.  There  are  thirty-five  criteria  in  the  C/SCSC 
Joint  Implementation  Guide  (4:1 1 -8-2-1).  These  criteria  define  the  standards  a 
contractor's  management  system  must  meet  regarding  organization,  planning 
and  budgeting,  accounting,  analysis,  revision  and  access  to  data. 

Cost  Performance  Reports  (CPR)  are  required  to  be  submitted  by  contractors 
on  ail  contracts  meeting  the  C/SCSC  dollar  thresholds  as  specified  in  DOO 
instruction  5000.2.  Cost  performance  reports  provide  current  and  oimulative 
cost  and  schedule  information  for  a  particular  contract.  On  smaller  contracts, 
C/SCSC  compliance  is  not  required  and  Cost/Scheduie  Status  Reports 
(C/SSR)  are  submitted  by  contractors  instead  of  the  CPR.  The  C/SSR 
contains  the  same  basic  information  as  a  CPR.  The  main  difference  between 
CPR's  and  C/SSR's  is  that  the  contractors  submitting  the  C/SSR's  do  not  have 
to  meet  the  requirements  of  a  full  C/SCSC-compiiant  management  system. 

Cost  Performance  Reports  and  C/SSR's  include  five  data  items  that  are 
the  cornerstones  of  CPR  analysis.  These  five  data  items  are:  ACWP,  BCWP, 
BCWS,  BAC  and  EAC  (4: 11-8-2-1). 

Actual  Cost  of  Work  Performed  (ACWP)  is  the  cost  incurred  and  recorded 
in  accomplishing  the  work  performed  within  a  given  time  period. 

Budgeted  Cost  of  Work  Performed  (BCWP)  is  the  sum  of  the  budgets  for 
completed  work  packages  and  completed  portions  of  open  work  packages. 
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plus  the  applicable  portion  of  the  budgets  for  level  of  effort  and  apportioned 
effort. 

Budgeted  cost  of  Work  Scheduled  (BCWS)  is  the  sum  of  budgets  for  all 
work  packages,  planning  packages,  etc.,  scheduled  to  be  accomplished 
(including  in-process  work  packages),  plus  the  amount  of  level  of  effort  and 
apportioned  effort  scheduled  to  be  accomplished  within  a  given  time  period. 

Budget  at  Completion  (BAG)  is  the  total  BCWS  for  the  entire  program.  It 
is  the  "spend  plan"  for  the  contract. 

Estimate  at  Completion  (EAC)  is  the  contractor's  estimate  of  the  total 
expected  costs  of  a  program.  EAC  is  the  current  ACWP  plus  the  budget  of  any 
work  that  still  needs  to  be  completed. 

Performance  data  flow  from  the  contractor  to  the  government  program 
offices  via  the  CPR  and  C/SSR,  which  are  collectively  known  as  cost 
management  reports.  This  data  are  used  by  the  program  offices  in  analysis 
and  reporting  to  upper  management.  One  such  report  is  the  quarterly  Defense 
Acquisition  Executive  Summary  (OAES)  report  that  is  submitted  by  major 
programs  to  the  Office  of  the  Under  Seaetary  of  Defense  (Acquisition)  where  it 
is  compiled  into  a  database  of  information  regarding  major  weapon  systems 
(20).  It  is  the  responsibility  of  the  Under  Secretary  of  Defense  (Acquisition)  in 
conjunction  with  the  Comptroller  of  the  Department  of  Defense  to  maintain  an 
automated  cost  data  segment  of  the  DAES  database.  This  database  is  the 
basis  from  ^\ch  the  research  In  this  thesis  is  drawn  (20:16-5). 

Information  from  cost  management  reports  has  been  used  in  a  variety  of 
ways  in  analyzing  the  progress  of  programs.  A  major  use  of  this  data  has  been 
along  the  lines  of  Estimate  at  Completion  (EAC)  analysis.  However,  it  is  also 
the  main  data  used  by  a  handful  of  researchers  who  have  applied  Rayleigh- 
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Norden  theory  in  the  analysis  of  program  growth.  The  next  sections  of  this 
chapter  will  examine  theory  involving  the  use  of  S-curves  in  the  analysis  of 
cost  growth  in  programs. 

Rayleigh-Norden  Curve  Theory 

Lord  Rayleigh  initially  developed  his  theory  to  explain  the  biological 
growth  patterns  of  plants.  The  growth  of  plants  as  a  percentage  of  growth  over 
time  resulted  in  a  bell  shaped  curve.  The  cumulative  representation  of  this 
curve  was  an  S-curve  which  modeled  the  growth  of  all  plants  (23).  This  pattern 
of  growth  was  later  extrapolated  to  represent  the  growth  of  all  systems  that 
followed  a  pattern  of  slow  initial  growth  followed  by  a  rapid  period  of  expansion 
in  the  middle  and  a  tapering  off  of  growth  at  maturity.  The  equation 
representing  this  S-curve  is  as  follows; 

(1) 

where 

1 .  Y  =  growth  during  each  period, 

2.  A:  =  total  cumulative  growth  of  the  plant, 

3.  a  =  shape  parameter  (governing  time  to  peak  growth). 

4.  t  s  elapsed  time  from  start  of  cycle. 

This  theory  was  extended  to  the  analysis  of  complex  projects  where  there 
is  a  pattern  of  manpower  buildup  and  phase-out  by  Norden  (24).  The 
Raylelgh-Norden  Curve  theory  or  the  S-Curve  was  central  to  the  research 
-tone  by  Weida  in  exploring  the  uses  of  the  S-Curve  as  a  cost  forecasting  tool 

v^O). 

Weida  developed  a  general  model  for  cost  forecasting  based  on  the 
expenditure  pattern  analysis  of  twenty-two  current  acquisition  contracts. 
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Weida's  data  all  came  from  CPR  reports  from  major  programs.  The  major 
problem  in  modeling  S-shaped  growth  curves  is  keeping  control  of  the  higher 
order  polynomials  required  to  express  the  complicated  S-shape.  Weida 
observed  that  S-curves  are  merely  derivatives  of  a  variety  of  bell  shaped 
curves,  each  of  which  has  an  inflection  point.  This  inflection  point  became  the 
division  where  Weida  broke  the  S-curve  in  half  and  developed  a  mathematical 
model  for  each  half  of  the  curve.  This  allowed  Weida  to  express  the  S-curve  in 
the  form  of  two  relatively  simple  logarithmic  or  quadratic  equations.  The  two 
mathematical  models  combined  represented  the  entire  growth  cycle  of  the 
project.  This  type  of  modeling  was  applied  to  fifteen  new  and  completely 
different  test  cases  with  good  accuracy,  demonstrating  that  the  S-Curves  could 
be  used  for  forecasting  the  costs  of  DOD  programs. 

Watkins  also  examined  the  use  of  S-Curves  in  estimating  project 
completion  costs  (13).  Watkins  used  the  Rayleigh-Norden  model  on  actual 
data  combined  with  regression  and  computer  analysis  to  forecast  total  contract 
costs.  Current  quarterly  ACWP  and  elapsed  time  were  the  input  parameters  to 
the  adopted  Rayleigh-Norden  model.  Watkins  converted  the  Rayleigh-Norden 

higher  order  polynomial  equation  into  a  linear  equation  to  facilitate  regression. 
Watkins  substituted  for  ^a,  where  was  the  time  of  peak  ACWP.  Both 

sides  of  the  equation  were  then  divided  by  /  and  transformed  Into  logarithmic 
equations.  The  resulting  equations; 


(2) 


were  in  the  familiar  linear  form  T  “  a  +  6x.  Watkins  used  regression  on  the 
first  4  quarters  of  contract  data  to  solve  the  equation  for  a  and  b.  These  were 
used  to  solve  for  T  which  in  Watkin's  thesis  became  the  next  quarter’s 
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incremental  ACWP.  When  the  next  quarter's  data  became  available  it  was 
plugged  into  the  equation  and  the  following  incremental  ACWP  was  then 
predicted.  Thus  a  prediction  of  each  subsequent  incremental  ACWP  was 
derived.  These  predicted  incremental  ACWP’s  were  compared  to  the  actuals 
for  accuracy.  The  result  was  a  less  than  1%  error  between  the  predicted  value 
and  the  actual  value  after  13  quarters  of  application  of  this  methodology. 

The  weakness  of  Watkins  methodology  is  that  by  the  time  you  got  to  the 
accuracy  of  the  13th  quarter  the  average  program  was  86%  complete.  At  that 
point  in  the  program  it  was  to  late  to  do  anything  about  correcting  the  program 
if  predictions  based  on  Watkin’s  methodology  showed  that  the  program  was  in 
trouble.  Finally,  Watkins,  while  drawing  from  a  sample  of  thirty  programs  in  his 
research,  only  used  three  programs  In  the  derivation  of  his  statistics.  This  is  a 
very  limited  sample  size.  However,  despite  the  weaknesses  in  Watkins 
methodology,  the  net  result  of  this  research  supported  the  use  of  Rayleigh- 
Norden  S-curves  In  the  prediction  of  cost  for  acquisition  programs. 

Table  1  summarizes  the  characteristics  of  the  studies  reviewed  for  this 

thesis. 


Table  1.  Relevant  Characteristics  of  Reviewed  Literature. 


AUTHOR 

YEAR 

«  OF  CONTRACTS 

CPROR 

C/SSR 

RESULTS 

WEIOA 

1977 

37 

CPR 

Supported  S^uirve 
analysis 

WATKINS 

1982 

3 

C/SSR 

Supported  S*cutve 
analysis 
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Another  use  of  the  S-Curve  In  DOD  program  analysis  involved  an 
adoption  of  Weida's  methodology  by  Mr.  Gary  Christie,  OUSD  (A)  (14).  Mr. 
Christie  plotted  the  percentage  of  cost  incurred  (ACWP/FINAL  ACWP)  against 
the  percentage  of  contract  completion.  Percentage  of  contract  complete  was 
calculated  as  percentage  of  time  to  that  point  over  total  time  as  measured  by 
the  final  report  date.  The  results  were  data  points  that  followed  an  "S"  shaped 
curve.  According  to  this  methodology,  Mr.  Wayne  Abba,  OUSD  (A),  states  that 
the  EAC  can  be  predicted  by  comparing  percentage  spent  and  the  percentage 
complete  of  a  contract  to  Its  expected  position  on  the  S-Curve(15).  The 
difference  between  the  contract  data  point  and  its  expected  point  on  the  S- 
Curve  gives  an  estimate  of  the  expected  cost  overrun  in  percentage  of  BAG. 
According  to  this  methodology,  if  the  observation  of  percentage  spent  for  a 
particular  program  is  significantly  different  from  the  S-curve  for  similar 
programs,  then  the  contract  will  not  be  completed  within  budget.  This 
methodology  was  used  by  OUSD(A)  as  evidence  that  the  A-12  program  was 
not  going  to  be  completed  within  budget  (3:2). 

This  chapter  examined  the  investigative  question  number  one,  "What 
research  has  examined  applying  S^cun/e  theory  to  OOD  cost  estimating?" 
Several  research  efforts  had  explored  the  application  of  $-cun/es  to  DOD  cost 
estimating.  Three  methodologies  were  examined  in  this  literature  review.  It  is 
the  methodology  of  Mr.  Christie  ^d  Mr.  Abba  which  was  pursued  in  this 
research. 

In  the  next  chapter  the  methodology  used  to  adapt  the  DAES  database  to 
S^urve  construction  for  analysis  of  programs  is  presented.  Subsequent 
chapters  present  the  results  of  this  research  and  discuss  conclusions  dravm 
from  the  research. 
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III.  Methodology 


Explanation  of  Method 

This  chapter  describes  the  steps  taken  in  manipulating  the  DAES 
database  for  the  purpose  of  determining  whether  S-curves  can  be  constructed 
which  are  appropriate  for  analyzing  cost  growth  on  DOD  projects.  The  second 
investigative  question  posed  in  this  thesis  asked  if  the  database  could  be  used 
to  construct  S  curves  similar  to  those  hand-aafted  by  OUSO(A)  in  its  analysis 
of  the  A-12  program  (Refer  to  Figure  1 ).  The  DAES  database  in  its  original 
form  did  not  contain  the  necessary  raw  data  to  construct  S-curves.  Additional 
data  fields  had  to  be  created.  In  order  to  accomplish  this  the  database  had  to 
be  reformatted  to  allow  for  mathematical  manipulation  of  the  raw  data.  The 
preliminary  step  in  explaining  the  methodology  used  in  this  thesis  will  be  a 
description  of  the  DAES  database. 

The  Database.  The  database  used  in  this  study  was  the  DAES 
database.  This  database  came  from  OUSO  (A)  and  contains  information 
covering  the  period  from  June  1970  to  November  1992.  The  DAES  database 
is  a  compilation  of  quarterly  information  provided  on  major  defense  programs 
or  other  programs  designated  for  Inclusion  by  the  Under  Secretary  of  Defense 
for  Acquisition.  The  information  in  the  DAES  database  comes  from  a  number 
of  sources  including  the  Comptroller  of  the  Department  of  Defense.  OSD  Cost 
Analysis  improvement  Group,  Service  Acquisition  Executives  and  the  Program 
Executive  Office.  The  primary  producer  of  the  quarterly  reports  that  feed  into 
the  database  is  the  program  manager  and  his  project  team.  The  project  office 
draws  on  the  information  provided  by  contractors  in  cost  performance  reports 
(CPR*s)  or  cost/schedule  status  reports  (C/SSR's)  to  complete  the  DAES 
reports. 
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The  initial  review  of  the  database  itself  revealed  that  the  database 
consists  of  three  separate  tables.  They  are:  1 )  Dcontract.dbf;  2)  Contract. dbf; 
and  3)  Histjano.dbf.  See  Tables  4,  5,  and  6  In  appendix  A.  Inspection 
revealed  that  the  format  of  many  of  the  fields  in  the  database  were  In  a 
character  format  versus  a  numeric  format.  This  did  not  lend  itself  to 
spreadsheet  manipulation  or  analysis.  The  appropriate  field  descriptions  were 
therefore  converted  to  numeric  format.  The  DAES  database  encompassed  a 
wide  variety  of  information,  most  of  which  was  not  required  for  the  purposes  of 
this  thesis.  Selection  of  the  fields  necessary  to  calculate  S-curves  developed 
by  OUSO(A)  were  identified.  In  order  to  calculate  percentage  of  cost  or 
percentage  spent,  fields  for  ACWP  and  TABUDG  were  required.  In  order  to 
calculate  percentage  of  time,  the  field  for  SUBMITOATE  was  required.  Only 
contracts  that  contained  records  over  the  period  before  25%  complete  to  at 
least  75%  complete  were  included  in  this  research  effort.  This  requirement 
stemmed  from  the  need  to  have  contracts  that  had  sufficient  data  points  to 
create  a  plot  of  a  complete  program.  To  calculate  percentage  of  contract 
completion  the  fields  for  BCWP  and  CBBASE  were  required.  Finally,  in  order 
to  sort  the  programs  by  phase,  branch  of  service,  type  of  program  and  contract 
type  the  fields  for  PPHASE,  WSDATE,  ECDATE,  OTBOATE,  CTYPE,  CDES, 
SERVICE.  PNASHORT,  and  PNALONG  were  required.  The  use  of  these 
fields  in  ^is  research  will  be  described  in  the  following  sections  of  this  chapter. 

The  DAES  database  was  reduced  to  those  fields  required  for  use  in  this 
research.  The  fields  required  are  identified  in  T^le  2. 
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Tab!©  2.  Required  Fields  from  the  Relevant  Databases  and  their  Descriptions. 


FIELD 

DESCRIPTION 

DATABASE 

PNO 

Program  Number 

DCONTRCT.DBF 

CONTRACT.DBF 

HIST_PNO.DBF 

CNO 

Contract  Number 

DCONTRCT.DBF 

CONTRACT.DBF 

SUBMITDATE 

Date  of  Report  Submission 

DCONTRCT.DBF 

CONTRACT.DBF 

PPHASE 

Program  Phase: 

Deveiopment  or  Production 

DCONTRCT.DBF 

WSDATE 

Work  Start  Date 

DCONTRCT.DBF 

BCWP 

Budgeted  Cost  of  Work  Performed 

DCONTRCT.DBF 

ACWP 

Aduat  Cost  of  Work  Performed 

DCONTRCT.DBF 

C68ASE 

Contract  Budget  Base 

DCONTRCT.OBF 

TABUDG 

total  Allocate  Budget 

DCONTRCT.DBF 

ECOATB 

E^tmaled  Completion  Date 

DCONTRCT.DBF 

OTBOATl 

Over  the  Budget  Aulhoriaation  Date 

DCONTRCT.DBF 

CTYPE 

Contract  Type: 

Co^  Plus  or  Fixed  Price 

CONTRACT.DBF 

DDES 

Contract  Description 

CONTRACT.DBF 

SERVICE 

Branch  of  Servir^: 

A«ARMY.  NsNAVY.  F*AIR  FORCE. 

MiST^PNODBF 

K*COAST  GUARD 


PNASHORT 

Short  Program  w  escdplion 

HtST.PNO.DBF 

PNALONG 

long  Program  Description 

HIST^PNO.OBF 
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In  order  to  manipulate  the  database,  each  record  had  to  be  uniquely 
identified.  The  identification  process  required  the  determination  of  a  suitable 
primary  key.  A  primary  key  for  a  database  is  used  to  uniquely  identify  an 
individual  record  within  each  table,  also  the  primary  key  should  be  consistent 
within  each  table.  The  primary  key  is  then  used  to  perform  key  manipulations 
of  the  database  such  as  sorting  and  joining  tables  as  well  as  retrieving  and 
deleting  specific  records.  The  field  that  uniquely  described  each  record  was 
defined  by; 


Primary  Key  =  PNO  ♦  CNO  ♦  SUBMITDATE. 

The  Ocontract.dbf  contains  59  fields  (See  Table  4,  appendix  A).  The 
Dajntraa  dbf  contained  15895  records  on  182  ptigrams  covered  by  1632 
different  contracts.  A  substantial  number  of  these  records  did  not  omtain 
sufficient  information  to  make  them  usable  and  were  deleted.  Generally  this 
consisted  of  records  with  zero  values  in  the  BCWP  or  ACWP  fields.  Deleting 
these  records  left  146  usable  programs  and  912  contracts.  The  Contract.dbf 
consisted  of  14  fields  (See  Table  5.  appendix  A).  The  Histjmo.dbf  contained 
7  fields  (See  Tables,  appendix  A).  The  stripped  down  DAES  database  which 
still  consisted  of  three  separate  tables  was  now  joined  into  one  table.  First 
Oa}ntra(^  was  joined  with  Contract  Uien  Uiis  combined  tabie  was  joined  with 
Hist j>no. 

Cie&tiirg  New  Fields.  The  database  was  inserted  into  an  Excel 
spreadsheet  so  that  the  numehcal  data  could  be  mathematically  manipulated. 
6CWP  was  modified  to  create  a  new  field  called  R6CWP  (revised  BCWP), 
desoibedas: 
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RBCWP  =  BCWP* 


CBPASEl 
TABUDG ; 


(3) 


V 


RBCWP  was  created  for  the  cases  where  the  contract  went  over  target 
budget  (which  happened  on  12  records).  An  Over  Target  Baseline  (OT3)  is  a 
Performance  Measurement  Baseline  (PMB)  resulting  from  formal 
reprogramming  by  the  contractor,  with  customer  approval,  which  establishes 
budget  allocation  in  excess  of  the  Contract  Budget  Base  (CBB). 

A  new  field,  CMPLT,  was  created  lo  determine  a  contract's  percentage  of 
completion.  It  is  described  as; 


^MPLT  = 


RBCWP 

(FINAL)  TABUDG 


(3) 


This  field  was  used  tr  determine  which  contracts  could  be  used  in  the 
development  of  our  graphical  analysis.  Only  those  contracts  tfiat  contained 
information  prior  lO  the  25%  completion  point  and  after  the  75%  complete  were 
desired.  Ir  order  to  effectively  model  S-curves  that  represented  the  life  of  a 
program  it  was  necessary  to  choose  data  that  represented  contracts  that  were 
nearly  complete.  Additionally,  these  contracts  had  to  have  records  that 
reflected  the  complete  history  of  the  program.  The  percentage  complete 
measurement  allowed  selection  of  those  contracts  that  provided  sufficient  data 
to  effectively  model  the  cost  growth  of  the  entire  contract. 

The  25%  to  75%  completion  criterion  had  some  limiting  effects  on  this 
research.  First  it  reduced  the  database  from  146  programs  and  912  contracts 
to  81  programs  and  312  contracts.  The  312  contracts  still  presented  a  large 
enough  database  to  examine  many  different  types  of  programs  by  category. 
Secondly,  the  reduced  database  eliminated  all  contracts  that  were  terminated 
prior  to  the  75%  cumpleiion  point.  Therefore  the  final  database  only 
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represented  successful  progranss.  This  was  not  considered  a  liability  because 
this  study  set  out  to  develop  S-curves  to  compare  against  current  programs. 
The  intent  was  to  develop  a  signature  S-curve  for  healthy  programs  and 
therefore  the  deletion  of  failed  programs  actually  aides  in  the  derivation  of  S- 
curves  that  are  representative  of  successful  programs. 

Initially,  the  current  value  of  TABUDG  was  used  to  determine  the  percent 
complete  instead  of  the  final  value.  However,  it  was  determined  that  all  other 
key  calculations  were  going  to  be  based  on  final  values,  therefore  we  choose 
to  use  the  final  reported  TABUDG. 

Two  other  fields  were  created  to  assist  in  the  refinement  of  the  database, 
they  were  MINCMPLT  and  MAXCMPLT.  These  fields  allowed  for  queries 
designed  to  extract  only  those  contracts  that  were  within  the  required  lim'ts  of 
percentage  complete.  They  simply  consisted  of  the  minimum  and  maximum 
values  of  percentage  complete  for  each  contract. 

Once  the  database  was  refined  to  Include  only  the  contracts  of  interest 
(25%  to  75%  complete),  two  other  fields  were  created  in  support  of  our  need  to 
graphically  represent  the  data  in  a  usable  format.  Specifically,  these  fields 
were  TIME  and  COST. 

TIME  represents  the  percentage  of  time  for  a  given  point  in  the  history  of 
contract  and  is  plotted  on  the  X  axis.  It  is  described  as: 

TIME  -  SUBMITDATE  -  First  SUBMITDATE) 

(Last  SUBMITDATE  -  First  SUBMITDATE)  '  ^ 

The  Y  axis  value  (COST)  is  the  percentage  of  costs  associated  with  a 
given  percentage  of  time  period,  it  is  described  as‘ 


COST  = 


Current  ACWP 
Final  TABUDG 


(5) 


18 


The  Final  Database.  The  final  database  consisted  of  41  fields  and  is 
described  in  Table  3. 


Table  3.  Final  Fields  Incorporated  into  the  Database  and  their  Descriptions. 


FIELD 

DESCRIPTION 

FIELD 

DESCRIPTION 

PNO 

Program  Number 

P_0 

Percent  of  Cost  at  0%  Time 

CNO 

Contract  Number 

P_5 

Percent  of  Cost  at  5%  Time 

SUBMITDATE 

Accounting  Period  Cut  Off  Date 

P_10 

Percent  of  Cost  at  10%  Time 

WSDATE 

Date  Work  Started  on  the 

Contract 

PJ5 

Percent  of  Cost  at  15%  Time 

ECDATE 

Estimated  Completion  Date 

Percent  of  Cost  at  20%  Time 

BCWP 

Cumulative  Budgeted  Cost  of 
Work  Performed 

P_25 

Percent  of  Cost  at  25%  Time 

RBCWP 

Revised  Budgeted  Cost  of  Work 
Performed 

P_30 

Percent  of  Cost  at  30%  Time 

ACWP  ' 

Cumulative  Actual  Cost  of  Work 
Performed 

P_35 

Percent  of  Cost  at  35%  Time 

TABUDQ 

Sum  of  Ail  Budgets  Allocated  to 
the  Contract 

P_40 

Percent  of  Cost  at  40%  Time 

MAXTAB 

Final  Total  Allocated  Budget 

P_45 

Percent  of  Cost  at  45%  Time 

PPHASE 

Program  Phase: 

Development  or  Production 

P.50 

Percent  of  Cost  at  50%  Time 

OTBDATE 

Over  Target  Baseline  Date 

P_55 

Percent  of  Cost  at  55%  Time 

CBBASE 

Sum  of  Negotiated  Cost  for  the 
Contract 

p.eo 

Percent  of  Cost  at  60%  Time 

ODES 

Contract  Description 

P_65 

Percent  of  Cost  at  65%  Time 

CTYPE 

Contract  Type: 

Cost  Plus  or  Fixed  Price 

P.70 

Percent  of  Cost  at  70%  Time 

SERVICE 

Branch  of  Service: 

AaARMY.  NaNAvY,  FaAIR 
FORCE.  K*COAST  GUARD 

P_75 

Percent  of  Cost  at  75%  Time 

PNASHORT 

Short  Program  Description 

om 

Percent  of  Cost  at  80%  Time 

PNALONG 

Long  Progfam  Description 

PJ5 

Percent  of  Cost  at  85%  Time 

CMPLT 

Percent  of  Contract  Completion 

om 

Percent  of  Cost  at  90%  Time 

MINCMPLT 

Minimum  Percentage  of  Contract 
mpletlon 

P.95 

Percent  of  Cost  at  95%  Time 

MAXCMPLT 

Ms>  lmum  Percentage  of  Contract 
Completion 

P.100 

Percent  of  Cost  at  100% 

Time 

TIME 

Percent  of  Time  Completion 

COST 

Percent  of  Cost  Expended 
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The  first  two  fields  are  the  program  number  (PNO)  and  contract  number 
(CNO)  assigned  by  OUSD(A)  to  identify  specific  programs  and  contracts 
contained  in  the  database.  The  fields:  SUBMITDATE,  BCWP,  RBCWP, 
ACWP,  TABUDG,  PPHASE,  and  CBBASE  are  all  described  in  the 
methodology  chapter  of  this  thesis.  WSDATE  is  the  date  the  work  started  on 
the  contract.  ECDATE  is  the  expected  completion  date  reported  by  the 
program  manager.  MAXTAB  is  the  largest  total  allocated  budget  reported  by 
any  of  the  quarterly  reports  for  a  specific  contract.  OTBDATE  is  the  date  the 
contract  exceeded  its  target  budget  if  this  situation  occurred.  ODES  contains 
a  brief  description  of  the  contract.  This  field  was  used  to  identify  program  type. 
CTYPE  identifies  the  contract  type.  The  SERVICE  field  identifies  the  branch  of 
the  service  managing  the  program.  CMPLT  is  the  percentage  of  completion  of 
the  program  for  a  particular  report  date.  Equation  3  describes  how  this  field 
was  calculated.  MINCMPLT  and  MAXCMPLT  represent  the  lowest  and  highest 
percentage  of  completion  reported  for  each  contract.  TIME  is  calculated  as  in 
equation  4,  and  COST  is  calculated  in  equation  5.  The  final  21  fields  all 
contain  the  normalized  percentage  of  time  from  0%  to  100%  in  5%  intervals  for 
each  contract. 

The  final  database  used  in  the  development  of  this  thesis  contained  81 
programs  composed  of  317  contracts.  The  contracts  averaged  15.35  records 
per  contract.  Generally  these  records  consisted  of  quarterly  reports.  However 
instances  of  monthly  reports  were  found.  The  database  is  derived  from  DAES 
reports  which  are  submitted  quarterly  as  directed  by  DOO  5000.2-4VI,  Section 
16.  The  average  contract  length  was  46.05  months  or  3.84  years.  The 
database  was  pared  down  to  include  only  programs  that  went  from  reporting 
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prior  to  or  on  25%  complete  to  at  least  75%  complete.  Therefore  the  length  of 
contract  is  representative  of  completed  contracts  since  it  excludes  contracts 
terminated  early.  The  longest  contract  contained  49  records  and 
encompassed  147  months  or  12.25  years.  The  average  number  of  contracts 
per  program  was  3.91  contracts.  Another  desaiptive  statistic  is  the  number  of 
contracts  that  went  over-  the-budget  (0TB).  These  are  contracts  that 
exceeded  the  target  budget  initially  established  for  the  program.  Interestingly, 
only  12  of  the  317  contracts  contained  in  the  database  were  0TB.  Of  these 
12,  two  were  Army  contracts,  and  10  were  Navy  contracts. 

The  earliest  record  in  the  database  was  6/25/70  and  the  latest  record  was 
3/25/91 .  The  database  encompassed  over  20  years  of  data  collection. 

Tables  7,  8,  and  9  in  appendix  B  provide  a  statistical  analysis  of  the  Total 
Allocated  Budget  (TABUDG),  the  Actual  Cost  of  Work  Performed  (ACWP).  and 
the  Estimated  Completion  Date  (ECD). 

At  this  point  a  substantial  portion  of  the  thesis  effort  was  completed. 

There  now  existed  a  database  that  could  be  used  to  develop  a  series  of  graphs 
displaying  an  S  curve  growth  pattern  of  percentage  of  cost  versus  percentage 
of  time  for  a  particular  subsection  of  the  database. 

Developing  the  Graphs.  An  example  of  how  the  database  can  be  used 
to  categorize  types  of  programs  is  now  presented.  Contracts  that  were 
associated  with  airframes  were  selected.  There  exist  in  the  database  8 
contact  for  airframes.  Specific  identification  of  contracts  and  or  contractors 
was  prohibited  OUSO(A).  An  initial  graph  was  plotted  using  the  raw  data  from 
the  database  (See  Figure  2). 
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Figure  2.  Graph  of  Airframes  using  raw  data  of  Time  versus  Cost. 

The  results  were  extremely  encouraging  because  the  airframe  contracts 
displayed  the  S-curve  anticipated  for  DOO  acquisition  programs. 

Normalization  of  the  database  was  required  to  obtain  equal  intervals  of  TIME 
for  all  contracts  so  that  a  group  of  contracts  could  be  averaged.  The  data  was 
normalized  to  5%  intervals  of  time.  To  normalize  the  data  the  TREND  function 
in  EXCEL  for  WINDOWS  was  used.  This  function  is  used  primarily  for  linear 
data.  To  use  the  TREND  function  in  this  situation  it  had  to  be  applied  to 
overlapping  ranges  of  data,  iteratively.  Even  this  method  did  not  truly 
automate  the  process.  An  extensive  portion  of  the  normalization  process 
involved  subjective  selection  of  cost  percentages  around  the  5%  completion 
intervals.  To  the  best  of  our  knowledge,  there  does  not  exist  a  function  within 
EXCEL  for  WINDOWS  that  will  fit  data  to  an  S  cun/e. 

To  validate  the  normalized  data,  sax  identical  plot  of  normalized  airframe 
data  was  overlaid  on  the  original  plot  of  non-normalized  airframe  data  (See 
Figure  3).  To  avoid  cluttering  the  graph,  only  three  contracts  were  used. 
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Figure  3.  Airframes  plotted  using  normalized  versus  raw  data. 


Visual  inspection  revealed  that  the  normalization  process  did  not 
materially  distort  the  plot.  In  instances  where  there  was  a  large  amount  of 
close  data  points,  the  normalized  plot  was  almost  an  exact  replica  of  the 
original  plot.  The  next  step  was  to  develop  Figure  4,  a  plot  showing  the 
average  and  two  standard  deviations  for  the  airframe  data. 
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Figure  4.  Airframe  data  averaged  plus  one  and  two  standard  deviations. 

Finally,  an  airframe  program  was  plotted  by  itself  as  an  overlay  against 
the  averaged  plot  with  the  standard  deviation  envelope.  Figure  5  is  a 
demonstration  of  how  a  program  manager  could  use  the  composite  S-curve  as 
a  tool  to  evaluate  the  progress  of  a  current  program.  The  program  would  be 
plotted  against  the  background  of  an  S-curve  constructed  from  the  average  of 
several  successful  programs  similar  to  the  current  program.  If  the  current 
program's  path  overlaid  the  S-curve  or  was  within  a  standard  deviation  its 
progress  would  be  verified  as  normal.  If  however  the  programs  growth  pattern 
was  outside  of  two  standard  deviations  of  the  S-curve  then  the  program  would 
need  to  be  examined  for  causes  of  the  atypical  behavior. 
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Figure  5.  Actual  contract  data  plotted  on  Airframes-Averaged. 

Sub-files  were  constructed  in  the  database  for  Army.  Navy  and  Air  Force. 
These  sub-files  were  constructed  to  make  it  easy  to  sort  categories  of 
programs  or  contracts  based  on  the  branch  of  the  service.  For  the  same 
reason  sub-files  were  created  for  cost  contracts,  fixed-price  contracts, 
production  contracts  and  development  contracts.  Additionally  sub-files  were 
created  for  the  different  types  of  programs  of:  airframe  and  helicopters.  These 
selected  sub-files  were  used  to  graph  categories  of  programs.  The  results  are 
presented  in  Chapter  4. 
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IV.  Results 


This  chapter  presents  a  detailed  description  of  the  final  database 
developed  during  this  research.  The  database  was  constructed  for  the 
development  of  S  curves  for  different  categories  of  defense  contracts.  A 
sample  of  the  various  S  curves  developed  from  the  database  will  also  be 
presented  In  this  chapter. 

Graphical  Results 

All  Programs.  Figure  6  is  a  plot  of  the  average  of  the  data  points  for  all 
81  programs  (317  contracts)  in  the  database.  The  anticipated  result  was  a 
composite  S  curve  for  DOD  programs.  Instead  the  plot  is  very  nearly  linear  in 
nature.  Envelopes  one  and  two  standard  deviations  wide  were  created  around 
this  composite  plot  of  the  percentage  of  cost  versus  the  percentage  of  time  for 
DOO  programs  sampled.  The  envelope  takes  on  more  of  the  shape  of  an  S 
curve  but  It  is  not  sharply  defined  and  It  is  very  broad  in  area.  At  the  50%  of 
time  point  a  two  standard  deviation  envelope  would  cover  programs  from  25% 
to  85%  complete.  That  is  a  difference  of  approximately  60%  complete.  Such  a 
wide  envelope  would  do  very  little  to  delineate  between  successful  and 
unsuccessful  programs. 

The  linear  nature  of  the  plot  of  the  composite  DOD  programs  graph 
possibly  represents  a  smoothing  out  of  the  S  curve  resulting  from  the 
variations  in  the  curves  of  the  different  contracts.  The  net  result  is  that  a  plot 
of  programs  against  this  composite  graph  could  not  be  used  as  intended.  A 
program  at  the  50%  completion  point  could  be  as  much  as  84%  spent  and  still 
be  within  two  standard  deviations  of  the  curve  derived  from  the  average  of  the 
programs.  Therefore  a  Program  Manager  in  this  situation  would  not  be  able  to 
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tell  from  a  comparison  plot  if  the  program  was  behaving  normally  or  was  over 
spent. 
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Figure  6.  Average  graph  of  all  programs. 


By  Service:  Air  Force,  Army  and  Navy.  Figures  7, 8  and  9  show  the 
averaged  and  standard  deviation  data  for  the  Air  Force,  Army  and  Navy 
respectively.  All  three  average  plots  resembled  a  linear  pattern  rather  than  the 
anticipated  S>curve.  Once  again  this  is  apparently  a  result  of  the  smoothing 
effect  tiiat  happens  when  a  composite  of  a  number  of  programs  is  graphed. 
Apparently  programs  the  r3.*inp  up  early  tend  to  balance  out  programs  that 
ramp  up  late.  The  result  is  the  smoothing  of  the  composite  S-curve  for  a 
category.  Another  interesting  observation  is  that  if  the  inflexion  point  of  the  S- 
curve  as  addressed  by  Weida  (10)  was  common  to  all  programs  then  there 
would  be  a  narrowing  of  the  envelope  at  the  inflection  point.  Since  this  does 
not  occur  it  is  an  indication  that  the  inflection  points  of  programs  may  vary 
among  programs. 
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Figures.  Navy  Contracts. 

Figure  10  provides  a  comparison  of  the  services  by  overlaying  the  graphs 
of  the  service  averages.  Very  little  differentiation  exists  between  Uie  services. 


%  OF  TIME 

Figure  10.  Comparison  of  Service  Averages. 

By  Contract  type:  Cost  Plus  and  Fixed  Price.  The  next  categories  of 
programs  plotted  were  sub  files  Uiat  contained  cost  plus  versus  fixed  price 


29 


contracts.  Figure  1 1  demonstrates  the  now  familiar  smoothed  out  curve  for 
both  types  of  contracts. 


Figure  1 1 .  Averages  of  Cost  Plus  versus  Fixed  Price  Contracts. 


8y  Program  Phase;  Development  and  Production.  Another  ms^or 
logical  division  of  programs  that  was  examined  was  development  programs 
versus  production  programs.  Figure  12  depicts  a  graph  of  the  average  of  ail 
development  contracts  versus  a  plot  of  the  average  of  all  production  contract. 
The  two  plots  were  overlaid  against  each  other  and  this  plot  revealed  little 
differentiation  between  the  program  phases.  Production  contracts  displayed 
more  of  a  weak  S  shape  Utan  did  development  programs. 


Figure  12.  Averages  of  Production  versus  Development  Contracts. 

By  Specific  Program;  Helicopters.  S  curve  anticipated  in  programs  vi^as 
established  in  theory  by  Rayleigh  and  Norden  and  demonstrated  by  Weida  and 
Watkins  in  their  research.  Figure  13  is  a  plot  of  10  helicopter  programs  and 
shows  the  anticipated  S  curve  shape  for  the  individual  programs. 


Figure  13.  By  specific  programs  using  Helicopters. 
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Figure  14  is  a  plot  of  the  average  for  the  helicopter  programs  and  shows 
the  smoothed  out  plot  exhibited  by  the  other  subdivisions  of  programs  when 
plotted  as  an  average.  In  this  plot  an  envelope  of  one  and  two  standard 
deviations  wide  is  graphed  around  the  average  plot  for  helicopter  programs. 
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Figure  14.  Average  and  Two  Std  Dev  for  Helicopter  Contracts. 

Next  Figure  15  plots  a  specific  helicopter  overlaid  on  the  graph  and  is 
compared  to  the  helicopter  envelope.  This  particular  program  appears  to  lie 
outside  two  standard  deviations  from  the  average  up  to  55%  of  time.  Without 
additional  history  on  this  program  no  explanation  of  the  deviation  is  attempted 
here.  This  graph  does  indicate  that  an  envelope  can  be  created  around  a  type 
of  program  for  comparison  with  a  specific  program  as  a  result  of  the 
development  of  tiie  database  constructed  during  this  research.  The  smoothing 
of  the  S  curve  due  to  averaging  mitigates  the  benefits  realized  from  such  a 
comparison  because  the  resulting  two  standard  deviation  envelope  is  so  large. 
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Figure  15.  Standard  Helicopter  Graph  with  an  Overlay 

Remarks..  This  chapter  has  presented  the  results  of  the  research 
undertaken  in  this  thesis.  A  database  was  developed  which  provided  the 
capability  of  plotting  percentage  of  cost  versus  percentage  of  time  readily  for  a 
variety  of  different  categories.  The  resulting  graphs  demonstrated  the  ability  of 
the  database  to  accomplish  this.  The  next  chapter  will  summarize  these 
results  and  describe  future  avenues  of  research  that  could  be  accomplished  by 
using  this  database. 
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V.  Conclusion 


A  Review  of  the  Hypothesis 

The  research  hypothesis  examined  was  a  two  part  hypothesis.  The  first 
part  was  that  the  DAES  database  could  be  used  in  the  creation  of  S  curve  plots 
of  programs.  This  part  of  the  hypothesis  needed  to  be  accomplished  first 
because  it  is  necessary  in  support  of  the  second  part  of  the  hypothesis  that  is 
that  S  curves  could  be  used  in  the  analysis  of  current  programs.  Six 
investigative  questions  were  addressed  in  exploring  this  thesis. 

1.  What  research  has  examined  applying  S-curve  theory  to  DOD 
cost  estimating? 

2.  Can  the  Defense  Acquisition  Executive  Summary  (DAES) 
database  be  used  to  produce  S  curves? 

3.  How  is  percentage  of  time  defined  and  derived? 

4.  How  is  percentage  of  cost  defined  and  derived? 

5.  Are  S  curves  different  for  various  categories  of  programs? 

6.  How  can  S-cufves  be  used  by  program  managers  and  analysts? 

Final  Analysis 

The  literature  review  explored  a  study,  a  thesis  and  an  application  of  the 
S>curve  technique  in  the  DOD  cost  estimating  environment.  Weida's  research 
approached  the  application  of  S-curve$  by  dividing  the  curve  at  the  inflection 
point  and  using  two  separate  quadratic  equations  to  model  the  S-curve.  This 
research  confirmed  the  S-curve  pattern  hypothesized  for  DOD  programs. 
Watkins  thesis  used  the  computer  to  calculate  S-curve  equations  for  a  limited 
number  of  programs  by  simplifying  the  equations  to  linear  form.  Watkins 
purpose  was  to  show  the  capability  of  the  computer  to  accomplish  such 
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analysis.  While  his  results  were  limited  in  scope  and  applicability  they  did 
demonstrate  that  DOD  acquisition  programs  display  the  S-curve  pattern  and 
that  this  pattern  can  be  predicted.  Finally,  Christie  and  Abba  of  OUSD(A)  used 
hand  calculated  S-curves  for  airframe  contracts  to  show  that  the  A-12  program 
did  not  display  the  typical  S-curve  demonstrated  by  other  successful  airframe 
programs.  Partially  based  on  this  analysis,  the  A-12  program  was  canceled. 
Thus  S-curves  have  been  explored  theoretically  for  use  in  analysis  of  DOO 
cost  estimates  and  practical  application  has  been  made  of  the  S-curve  theory. 

This  thesis  was  based  on  the  belief  that  a  database  could  be  used  to 
graph  S-curves  for  various  categories  of  programs.  To  this  end  the  DAES 
database  was  secured  from  OUSD(A)  and  explored  as  a  potential  source  of 
the  type  of  information  required  for  development  of  S-cun/es.  After 
manipulation  of  the  database,  S-curves  were  successfully  developed  for  single 
programs  and  categories  of  programs.  The  composite  curves  developed  from 
the  averages  of  several  programs  all  exhibited  a  near  linear  form.  This 
smoothing  of  the  S-curve  was  unanticipated  and  apparently  was  the  result  of 
the  averaging  of  the  individual  S-cun/es. 

The  third  and  fourth  investigative  questions  addressed  the  mechanics  of 
constructing  S-curves.  They  dealt  with  the  derivation  of  percentage  of  cost 
and  percentage  of  time.  Percentage  of  Cost  was  calculated  as  current  ACWP 
divided  by  Final  TABUDG  (See  Equation  5).  Percentage  of  Time  was 
calculated  as  current  SUBMITOATE  minus  First  SUBMITDATE  divided  by  Last 
SUBMITDATE  minus  First  SUBMITDATE  (See  Equation  4). 

The  fifth  investigative  question  asked  if  different  S-curves  were  produced 
for  different  categories  of  programs.  The  S-curves  produced  by  averaging 
categories  of  programs  were  nearly  linear  in  form  for  all  categories  of 
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programs.  The  averaging  effect  of  a  number  of  programs  apparently  resulted 
in  the  smoothing  out  of  the  curve.  As  a  result  the  graphs  produced  for  various 
categories  of  programs  were  nearly  identical  because  they  were  all  nearly 
linear. 

Finally,  the  last  investigative  question  asked  how  S-curves  could  be  used 
by  program  managers  and  analysts.  The  original  premise  of  this  thesis  was 
that  composite  S-curves  could  be  developed  for  categories  of  programs  and 
program  managers  and  analysts  could  plot  their  programs  of  interests  against 
these  plots  to  see  how  they  compared.  However,  the  resulting  composite 
cunres  were  so  linear  in  nature  that  the  did  not  provide  a  good  basis  for 
comparison  to  current  programs. 

The  first  part  of  the  hypothesis  was  accomplished  in  that  a  modified 
database  was  constructed  from  the  DAES  database  and  it  can  be  used  to 
readily  accomplish  a  variety  of  ways  of  examine  programs  based  on 
comparison  of  percentage  of  time  versus  percentage  of  cost.  The  second  part 
of  the  hypothesis  was  not  accomplished  in  this  thesis.  A  plot  of  the  average  of 
a  number  of  programs,  no  matter  how  divided,  presents  more  of  a  linear  plot 
than  the  anticipated  S  curve  that  could  be  meaningfully  compared  to  active 
programs.  Tfiis  linear  appearance  results  from  the  averaging  of  different 
programs  with  varying  S  cun/e  signatures  in  arriving  at  a  graph  of  the  average 
of  a  number  of  programs.  A  comparison  of  this  type  of  linear  plot  to  a  specific 
program  is  not  particularly  revealing  or  valuable. 

Recommendations  for  Further  Research 

Time  did  not  allow  this  research  to  fully  examine  the  potential  uses  of  the 
database  derived  from  the  DAES  database.  The  primary  approach  used  to 
examine  programs  was  to  plot  different  cuts  of  the  averages  of  programs  by 
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category.  These  averages  did  not  provide  the  S  curve  signature  anticipated  of 
a  plot  of  programs  percentage  of  cost  versus  percentage  of  time.  However, 
another  possible  approach  to  analyzing  specific  programs  would  be  plot 
them  against  analogous  successful  programs.  A  development  program  could 
be  plotted  against  other  successful  development  programs,  not  against  an 
average  of  these  programs.  Standard  deviation  envelopes  could  be 
constructed  around  the  individual  successful  programs  to  provide  an  indication 
of  degree  of  divergence  of  the  program  under  examination  from  the  baseline 
program.  This  avenue  of  comparison  may  prove  valuable  as  an  analytical  tool 
for  program  managers. 

The  primary  product  resulting  from  this  research  is  tne  manipulated  DAES 
database.  This  database  offers  many  different  venues  of  future  exploration.  It 
provides  the  capability  of  examining  program  data  from  a  variety  of  different 
aspects,  each  of  which  could  lead  to  future  research  topics. 
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Appendix  A:  Original  OAES  Table  Stmctures 


Field 


PNO 


CNO 


IDCLASS 


CTRDIV 


PPHASE 


CHANGENO 


TOTQTY 


PLANQTY 


DELQTY 


NEGCST 


AUWORK 


TPRICE 


CPRICE 


WSDATE 


SECDATE 


RPD 


SOOC 


RTYPE 


RDATE 


BOWS 


BCWP 


ACWP 


MR 


C86ASE 


TABUDQ 


CTREST 


PMEST 


ECDATE 


OTBOATE 


OTBCVA 


OTBSVA 


COMMENTS 


COMCUSS 


CRITNM1 


Table  4.  Dcontrctdbf  table  desaiption. 


Field  Name 


PROGRAM  NUMBER 


CONTRACT  NUMBER 


SUBMISSION  DATE 


UNIQUE  IDENTIFIER 


CONTRACTOR  DIVISION 


PROGRAM  PHASE:  PRODUCTION  OR 
DEVELOPMENT 


CHANGE  ORDER  NUMBER 


TOTAL  CONTRACT  QUANTITY 


PUNNED  TO  DATE  DELIVERY  QUANTITY 


ACTUAL  QUANTITY  DELIVERED  TO  DATE 


NEGOTIATED  COST 


COST  OF  AUTHORIZED.  UNPRICED 
WORK 


TARGET  PRICE 


CEILING  PRICE 


DATE  WORK  STARTED 


SIGNIFICANT  EFFORT  COMPLETION 
DATE 


REPORT  DATE 


SOURCE  DOCUMENT:  CPR.  C/SSR, 
OTHER 


REVIEW  TYPE 


REVIEW  DATE 


BUDGETED  COST  OF  WORK  SCHEDULED 


BUDGETED  COST  OF  WORK 
PERFORMED 


ACTUAL  COST  OF  WORK  PERFORMED 


MANAGEMENT  RESERVE 


CONTRACT  BUDGET  BASE 


TOTAL  ALLOCATED  BUDGET 


CONTRACTOR  ESTIMATED  COST 


PROGRAM  MANAGER  ESTIMATED  COST 


ESTIMATED  COMPLETION  DATE 


OVER  THE  BUDGET  DATE 
AUTHORIZATION 


OVER  THE  BUDGET  COST  VARIANCE 


OVER  THE  BUDGET  SCHEDULE 
VARIANCE 


PROGRAM  MANAGER  COMMENTS 


Width 


CRITICAL  MILESTONE  1 


CHARACTER 


CHARACTER 


DATE 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


DATE 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


Mssmsimi 


CHARACTER 


CHARACTER 


CHARACTER 


CHARACTER 


MEMO 


CHARACTER 


CHARACTER 
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Table  4.  Dcontrct.dbf  table  description  (continued). 


Field 

Field  Name 

Bmm 

Width 

^Rn>lM2 

CRITICAL  MILESTONE  2 

CHARACTER 

20 

CRITD1  ' 

CRITICAL  MILESTONE  1  DESCRIPTION 

CHARACTER 

6 

CRITD2 

CRITICAL  MILESTONE  2  DESCRIPTION 

CHARACTER 

6 

VARCLASS 

VARIANCE  CLASSIFICATION 

CHARACTER  1 

1 

CVBASE  I 

COST  VARIANCE  DATA 

CHARACTER 

8 

CVUCER 

COST  VARIANCE  DATA 

CHARACTER 

8 

CVCUR 

COST  VARIANCE  DATA 

CHARACTER 

8 

CVBASEPER 

COST  VARIANCE  DATA  1 

CHARACTER 

7 

CVUCERPER 

COST  VARIANCE  DATA  ' 

CHARACTER 

7 

CVCURPER 

COST  VARIANCE  DATA 

CHARACTER 

7 

SVBASE 

SCHEDULE  VARIANCE  DATA 

CHARACTER 

8  ! 

SVUCER 

SCHEDULE  VARIANCE  DATA 

CHARACTER 

8  1 

SVCUR 

SCHEDULE  VARIANCE  DATA 

CHARACTER 

8 

SVBASEPER 

SCHEDULE  VARIANCE  DATA 

CHARACTER 

7 

SVUCERPER 

SCHEDULE  VARIANCE  DATA 

CHARACTER 

7 

SVCURPER 

SCHEDULE  VARIANCE  DATA 

CHARACTER 

7 

CBDATE 

CONTRACT  BUDGET  BREACH  DATE 

DATE 

8 

CBBREACH 

CONTRACT  BUDGET  BREACHED 

LOGICAL 

1 

PMBEST 

PROGRAM  MANAGERS  BEST  ESTIMATE 

CHARACTER 

8 

PMWORST 

PROGRAM  MANAGERS  WORST 

ESTIMATE 

CHARACTER 

8 

BASEDATE 

DATE 

1  8 

OSDNOTES 

OSD  NOTES 

MEMO 

1  10 

OSDCUSS 

OSD  CLASSIFICATION 

CHARACTER 

1 

CAEST 

CHARACTER 

8 
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Table  5.  Contract.clbf  table  description. 


Field 

Field  Name 

Width 

PNO 

PROGRAM  NUMBER  1 

CHARACTER 

SB 

CNO 

CONTRACT  NUMBER 

CHARACTER 

SUBMITDATE 

SUBMISSION  DATE  ! 

DATE 

8 

COES 

CONTRACT  DESCRIPTION 

CHARACTER 

24 

CID 

CONTRACT  IDENTIFICATION  NUMBER  ■ 

CHARACTER 

16 

CTYPE 

CONTRACT  TYPE  -  FIXED  OR  COST 

CHARACTER 

7 

CTRNAME 

CONTRACTORS'  NAME 

CHARACTER 

25 

CTRCITY 

CONTRACTORS'  CITY 

CHARACTER 

20 

CTRSTATE 

CONTRACTORS'  STATE 

CHARACTER 

2 

CTRZIP 

CONTRACTORS'  ZIP  CODE 

CHARACTER 

10 

COMPLETE 

CONTRACT  COMPLETE 

LOGICAL 

1 

DEFINED 

DATE 

8 

MAJOR 

LOGICAL 

1 

RIC 

CHARACTER 

1 

Table  6.  Histjano.dbf  table  description. 


Field 

Field  Name 

Type 

Width 

PNO 

PROGRAM  NUMBER 

CHARACTER 

3 

PNASHORT 

SHORT  PROGRAM  DESCRIPTION 

CHARACTER 

10 

PNA 

PROGRAM  DESCRIPTION 

DATE 

20 

PNALONG 

LONG  PROGRAM  DESCRIPTION 

CHARACTER 

70 

SERVICE 

BRANCH  OF  SERVICE:  F  •  AIR  FORCE. 

A  -ARMY.  N  •  NAVY.  K  -  COAST  GUARD 

CHARACTER 

1 

CURRENT 

LOGICAL 

1 

APB_NUMBER 

CHARACTER 

4 
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Appendix  B:  Descdptive  Statistics  of  the  database 


Alfocated  Budget  Mean  and  Standard 


Table  8.  Statistical  Analysts  of  Actual  Cost  of  Work  P3rform8d  (ACWP). 
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Figure  17.  Actual  Cost  of  Work  Performed  Mean  and  Standard  Deviation. 
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